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ABSTRACT 


In  this  paper  we  present  a  hierarchical  loss  network  model  for  estimating  the  end-to-end 
blocking  probabilities  for  large  networks.  As  networks  grow  in  size,  nodes  tend  to  form 
clusters  geographically  and  hierarchical  routing  schemes  are  more  commonly  used.  Loss 
network  and  reduced  load  models  are  often  used  to  approximate  end-to-end  call  blocking- 
probabilities  and  hence  throughput.  However  so  far  all  work  being  done  in  this  area  is  for 
fiat  networks  with  flat  routing  schemes.  We  aim  at  developing  a  more  efficient  approximation 
method  for  networks  that  have  a  natural  hierarchy  and/or  when  some  form  of  hierarchical 
routing  policy  is  used.  We  present  two  hierarchical  models  in  detail  for  fixed  hierarchical 
routing  and  dynamic  hierarchical  routing  policies,  respectively,  via  the  notion  of  network 
abstraction,  route  segmentation,  traffic  segregation  and  aggregation.  Computation  is  done 
separately  within  each  cluster  (local)  and  among  clusters  (global),  and  the  fixed  point  is 
obtained  by  iteration  between  local  and  global  computations.  We  also  present  numerical 
results  for  the  first  case. 


‘Prepared  through  collaborative  participation  in  the  Advanced  Telecommunications/Information  Distri¬ 
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1  Introduction 


Modern  military  networks  are  constructed  by  integrating  satellite,  wireless,  Internet  and  ad 
hoc  technologies.  They  tend  to  be  both  hybrid  and  hierarchical.  Correspondingly,  routing- 
schemes  are  also  becoming  increasingly  hierarchical  in  order  to  scale  up  with  the  size  of  the 
network.  Consider  a  typical  military  network  scenario:  think  of  the  soldiers  as  the  bottom 
layer  of  the  communication  hierarchy,  the  HUMVEEs  as  the  second  layer  and  the  satellites 
as  the  top  layer.  When  a  soldier  establishes  connection  with  a  remote  soldier,  the  call  is 
routed  first  through  the  HUMVEE,  which  serves  as  gateway  for  a  group  of  soldiers  (wireless 
LAN).  The  HUMVEE  may  decide  whether  to  route  the  call  via  the  satellite  -  further  up  in 
the  hierarchy,  or  to  another  HUMVEE  on  the  same  layer. 

We  consider  a  class  of  loss  network  models  [1].  Extensive  research  has  been  done  in  using 
reduced  load/  fixed  point  approximations  to  estimate  call  blocking  probabilities,  which  is 
the  primary  performance  metric  of  circuit  switched  networks.  With  the  development  of  QoS 
routing  and  ATM  networks,  the  same  technology  of  reduced  load  approximation  has  been 
applied  to  packet  switched  networks  for  connection  level  study  via  the  concept  of  effective 
bandwidth  [2], 

While  research  results  are  abundant  for  fully  connected,  symmetric  networks  with  fixed, 
sequential  or  state-dependent  routing  [1],  esp.  for  networks  with  no  more  than  two  hops 
on  their  routes  [3],  or  when  network  traffic  is  of  single  rate  [4],  there  has  been  far  less 
attention  to  large  random  networks  with  both  multiple  traffic  rates  and  state-dependent 
routing  [1,  5,  6,  7].  Furthermore,  all  of  such  methods  are  for  flat  networks  and  fiat  routing- 
schemes. 

Motivated  by  the  increasing  frequency  of  the  occurrence  of  large  randomly  or  sparsely 
connected  hierarchical  networks,  we  develop  a  hierarchical  version  of  the  reduced  load  model. 
We  examine  two  types  of  hierarchical  routing  schemes  and  the  corresponding  end-to-end 
connection  level  models.  One  is  fixed  or  near  fixed  routing  with  the  typical  example  being 
OSPF,  which  is  widely  used  for  Internet,  IP  based  routing.  Under  this  routing  scheme, 
routes  are  established  based  on  the  shortest  distance  principle,  with  ties  broken  according  to 
lower  IP  address.  Considering  the  fact  that  links  normally  fail  on  a  much  larger  time  scale 
compared  to  connection  durations,  this  is  a  fixed  routing  scheme. 

The  other  type  is  dynamic/state  dependent/adaptive  hierarchical  routing  with  the  typical 
example  being  PNNI.  Various  proposals  for  QoS  routing  in  the  Internet  also  fall  under  this 
category  [8,  9].  In  this  case,  the  centering  point  is  “partial  information”.  Networks  are 
divided  into  clusters  or  peer  groups.  Each  non-border  nodes  is  only  aware  of  its  own  peer 
group.  Routes  are  established  on  different  layers  based  on  complete  information  within  a 
peer  group  and  aggregated  information  between  peer  groups.  The  advantage  of  having  a 
hierarchical  end-to-end  model  is  that  it  closely  corresponds  to  the  hierarchical  nature  of 
routing  and  uses  only  partial  information  on  different  layers. 


2 


Substantial  numerical  experiments  are  in  progress.  In  the  next  section  we  describe  net¬ 
work  abstraction  and  aggregation.  Hierarchical  models  for  fixed  hierarchical  routing  and 
dynamic  hierarchical  routing  are  presented  in  Section  3  and  4,  respectively.  In  Section  5  we 
present  preliminary  numerical  results  for  the  fixed  hierarchical  routing  case,  which  gained 
approximately  4-fold  improvement  in  computational  cost.  Section  5  concludes  the  paper. 


2  Network  Abstraction 


We  only  consider  large  networks  that  have  either  physical  hierarchies  or  routing  hierarchies 
vs.  a  complete  mesh  since  a  hierarchical  model  promises  clear  incentives  only  for  the  former 
even  if  it  is  at  all  possible  for  the  latter.  Throughout  the  paper  we  will  be  using  a  two-layer 
example  shown  in  Figure  1. 


1.2 


Figure  1:  Network  with  three  peer  groups  -  Layer  One 


There  are  three  peer  groups  in  this  example,  with  the  dash-circles  surrounding  each  one. 
Each  group/node  has  an  address.  All  border  nodes  are  shown  in  black.  A  non-border  node 
does  not  necessarily  have  a  direct  link  connected  to  border  nodes,  although  this  is  often  true 
with  IP  networks.  Note  that  all  links  on  this  layer  are  actual,  physical  links. 

All  border  nodes  are  kept  in  the  higher  layer  -  in  this  case  Layer  2  and  border  nodes 
belonging  to  the  same  peer  group  are  fully  connected  via  “logical  links” ,  illustrated  in  dashed 
lines  if  they  do  not  correspond  to  a  physical  link  as  shown  in  Figure  2. 

As  pointed  out  in  [10],  creating  a  logical  link  between  each  pair  of  border  nodes  is  the 
full-mesh  approach,  while  collapsing  the  entire  peer  group  into  a  single  point  is  the  symmetric- 
point  approach.  Our  aggregation  approach  is  a  full-mesh  one.  While  it  may  not  be  the  most 
economic  way  of  aggregation,  this  model  clearly  reflects  best  the  underlying  network  physical 
structure  and  routing  scheme.  It’s  worth  pointing  out  that  a  bandwidth  parameter  is  usually 
assigned  to  a  logical  link,  e.g.,  representing  the  maximum/average  available  bandwidth  on 
the  paths  between  two  border  nodes,  and  this  may  cause  problems  when  different  paths 
overlap  [9].  However,  as  we  will  see,  bandwidth  is  not  necessarily  the  parameter  in  our 
model  for  calculation  on  the  higher  layer,  thus  avoiding  the  aforementioned  problem.  As 
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1.2.2 


Figure  2:  Network  with  three  peer  groups  -  Layer  Two 


described  in  detail  in  later  sections,  in  the  fixed  routing  case,  this  parameter  is  the  blocking 
probability  which  resulted  from  previous  iterations  within  the  peer  group,  and  in  the  dynamic 
routing  case,  this  parameter  can  be  implied  costs,  hop  number  or  other  criteria  based  on  the 
dynamic/QoS  routing  policies  being  used. 


3  Hierarchical  Model  for  Fixed  Routing 

3.1  Notations 

G(l.n):  the  nth  cluster/peer  group  on  Laver  1,  where  n  =  l,...,iVi,  and  N\  is  the  total 
number  of  peer  groups  in  Layer  1. 

l.n.x:  node  x  in  peer  group  G(l.n),  where  x  =  1, ...,  Xn,  and  Xn  is  the  total  number  of 
nodes  in  G(l.n). 

l.n.y:  border  nodes  in  peer  group  G(l.n). 

l.n.Xi  — >  l.n.x 2'-  link  from  node  l.n.Xi  to  node  l:n.x2.  Links  in  our  model  are  direc¬ 
tional. 


As(l.ni.a’i  — >  l.n2.x2):  offered  load  for  class- s  traffic  from  source  l.n\.x\  to  destination 
1.U2-X2,  where  s  =  1  ,...,5,  and  S  is  the  total  number  of  different  traffic  classes.  It  is  also 
written  as  Xps  with  p  as  the  pth  source-destination  node  pair. 

V{l.ni.x\  — >  l.n2.x2).  the  route  set  between  node  l.ni-.Xi  and  l.n2.a;2.  Vp  is  the  route 
set  for  the  pth  node  pair. 
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3.2  Routes  and  Route  Segments 


For  our  modeling  purposes,  each  route  is  broken  down  into  route  segments  whenever  a  route 
goes  across  the  border.  Therefore,  a  typical  route  V(l.n\.x\  — >  l.n2.:r2)  is  segmented  into 
the  following  k  segments,  assuming  that  n\  ^  n2  and  that  neither  (l.n\.X\)  nor  (l.n2.:r2)  is 
a  border  node: 


V1(l.nl.x1  — »  l.ni.j/12)  'P2(l.n1.i/i2  — >  l-rii-yn) 

V3{1.  rti.yn  — >  1. n,;.y,;2)...  Vk{l.n2.y2i  — >  l.n2.x-2) 

where  the  subscript  in  y}  \  indicates  this  is  a  border  node  from  which  traffic  comes  into  peer 
group  j,  and  y/2  indicates  this  is  a  border  node  from  which  traffic  leaves  peer  group  j.  We 
denote  the  set  of  route  segments  for  the  pth  source-destination  node  by  Vp. 


3.3  Initial  Offered  Load  and  Local  Relaxation 

The  offered  load  of  class-.s  traffic  of  the  pth  node  pair  (l.ni.ay,  l.n2.a;2)  is  A^l.ni.ay  — > 
1  .n2.x2).  We  substitute  this  with  a  combination  of  the  following,  in  a  similar  way  as  route 
segmentation: 


X°ps{1.7li.Xi  — 

->■  l-ni.yi2) 

src.  peer  grp. 

A°s(l.ni.yi2  - 

->■  1  -rh-yn) 

inter-peer  grp. 

A|!,i  1  •//,•//,!  - 

-t  1  .rii.ya) 

intermediate  peer  grp.  i 

A°s(l.n2.y2i  - 

l.n2.a;2) 

des.  peer  grp. 

These  terms  all  take  on  the  value  of  the  real  offered  load  \Qs(l.n\.X\  — >  l.n2.:r2).  Thus  we 
have  complete  input  information  (together  with  route  segments)  for  each  peer  group.  For 
the  %th  peer  group,  offered  loads  indexed  with  same  node  pairs  are  added  up  to  represent  the 
aggregated  traffic.  Here  we  assume  that  at  least  one  of  the  nodes  is  a  border  node  since  no 
such  additional  process  is  necessary  with  the  case  where  both  nodes  are  non-border  nodes 
within  the  same  group.  Without  loss  of  generality,  assume  that  the  destination  node  is  a 
border  node. 


X](l.ni.xi  — >  1  .rii.ya)  = 

J2  \°ps{l.nt.xi  — >  l.nt.yi2). 

{p:V(l.tH.xi — >1  -ni-y^eVp} 

The  fixed  point  method  is  then  applied  to  every  peer  group  separately  using  these  offered 
loads  to  calculate  group-wide  end-to-end  blocking  probabilities:  Bs(l.ni.xi  — >  l.ni.yi2), 
Bs(l.ni.yn  — >  l.n,.y,2).  £?s(l.n2.y2i  — >  l.n2..T2).  By  doing  so,  the  initial  condition  of  the 
algorithm  is  chosen  to  be  of  zero  inter-group  blocking. 
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3.4  Reduced  Load  and  Higher  Layer  Relaxation 


On  the  higher  layer  (second  layer  in  our  example),  only  border  nodes  exist.  We  construct  a 
new  network  with  border  nodes,  inter-group  links  and  logical  links  as  illustrated  in  Figure 
2.  For  this  logical  network  we  have  the  following  offered  load: 

Aj(l.ni.yi2  — >  l-n2.y2i)  =  A°(l.ni.?/i2  — >  l.n2.y2i)  + 

Y  A°s( l.n^yn  — >  l.n2.y21)  • 

{p:V(l.ni.yl2 — >l.n2.H2l)e'Pp} 

Bs{l.n1.x1  — >  l-ni.yl2)  ■  Bs(l.n2.y21  — >  l.n2.a:2), 

This  is  the  initial  offered  load  thinned  by  blocking  in  both  the  source  and  destination  peer 
groups.  For  simplicity  purposes,  we  use  V{l.n\.y\2  — >  l.n2.y2i)  for  either  a  single  route 
segment  or  combination  of  multiple  route  segments  belonging  to  the  same  route. 

We  again  apply  the  fixed  point  approximation  to  this  layer  and  calculate  second-layer 
end-to-end  blocking  probabilities.  The  result  of  this  step  is  the  group-to-group  blocking- 
probability:  Bs(l.ni.y12  — »  l.n2.y21). 


3.5  Iteration 

Using  the  results  from  the  inter-group  approximation,  replace  the  offered  load  with: 

Aj(l -Ui-xx  ->■  1  .rii.ya)  =  Y  A°s(l.n*.;ri  ->■  • 

{p-.AeVP} 

Ba{l.rii.y&  ->■  1  //j i )  •  Ba{l.nj.yjX  ->■  1  .rij.x2). 

where  A  is  defined  as  the  union:  V{1  .rij.Xi  — )■  l.niAyi2)lJ'P(l.n;ryi2  — )■  l.nj.yji)UV{l.rij.yji  —)■ 
l:rij.x2).  This  is  essentially  the  original  offered  load  thinned  by  blocking  on  inter-group  links 
and  the  remote  peer  group.  This  becomes  the  new  input  for  local  relaxation.  Local  and 
higher  layer  relaxations  are  then  repeated  till  the  difference  between  results  from  successive 
iterations  are  within  certain  criteria. 


4  Hierarchical  Model  for  Dynamic  Routing 


There  are  numerous  existing  and  proposed  dynamic/QoS  hierarchical  routing  schemes,  each 
of  which  results  in  different  end-to-end  performances  determined  by  the  scope  and  design 
trade-off  of  the  routing  scheme.  Our  primary  goal  here  is  not  to  design  an  end-to-end 
model  for  each  and  everyone  of  these  schemes.  Rather,  we  attempt  to  present  an  end-to-end 
performance  modeling  framework  that  considers  a  “generic”  type  of  dynamic  hierarchical 
routing,  which  captures  some  of  the  most  basic  properties  of  a  majority  of  such  routing 
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schemes.  We  make  assumptions  for  simplicity  purposes,  but  our  work  shows  how  an  end-to- 
end  performance  model  can  be  closely  coupled  with  routing  policies  to  provide  an  efficient 
way  of  analysis.  Furthermore,  our  model  enables  us  to  analyze  situations  where  different 
routing  schemes  are  used  on  difference  levels  of  a  networks. 


4.1  Dynamic  Hierarchical  Routing 

One  key  property  of  any  dynamic  hierarchical  routing  is  inaccurate/incomplete  information. 
A  node  has  only  aggregated  information  on  other  peer  groups  advertised  by  the  border  nodes. 
This  aggregated  information  can  be  one  or  more  of  various  metrics  specified  by  the  routing 
algorithm:  implied  cost  of  a  peer  group  maximum  available  bandwidth  between  border  node 
pairs,  end-to-end  blocking  or  delay  incurred  by  going  through  a  peer  group,  etc.. 

In  source  routing,  a  path  is  selected  with  detailed  hop-by-hop  information  in  the  originat¬ 
ing  peer  group  but  only  group-to-group  information  beyond  the  originating  group.  The  choice 
of  routes  within  a  group  can  be  determined  using  shortest  path  routing,  least  loaded/state 
dependent  routing  and  so  on.  The  routes  between  groups  are  primarily  determined  by  the 
form  of  aggregation  advertised  by  border  nodes.  A  call  is  blocked  if  the  route  selected 
according  to  the  dynamic  routing  policy  does  not  have  the  required  bandwidth. 


4.2  Probabilistic  Offered  Load  Distribution  and  Higher  Layer  Re¬ 
laxation 

One  of  the  main  advantages  of  dynamic  routing  is  “load  averaging”,  i.e. ,  dynamically  dis¬ 
tribute  traffic  flow  onto  different  paths  of  the  network  to  achieve  greater  utilization  of  net¬ 
work  resources.  We  argue  that  under  steady  state,  a  particular  traffic  flow  (defined  by  class, 
source-destination  node  pair)  is  distributed  among  all  feasible  routes,  and  among  multiple 
border  nodes  that  connect  to  other  peer  groups.  (This  problem  does  not  exist  when  there 
is  only  one  border  node.  Routes  are  still  dynamically  chosen,  but  all  routes  ultimately  go 
through  that  single  border  node.)  The  fraction  of  a  traffic  flow  that  goes  through  a  certain 
border  node  is  directly  related  to  the  aggregated  information/metrics  for  the  group-to-group 
route  the  border  node  sees. 

Based  on  this,  for  a  pair  of  nodes  belonging  to  different  clusters,  the  feasible  route  set 
are  divided  into  three  subsets:  routes  within  the  source  peer  group,  routes  between  groups 
and  routes  within  the  destination  peer  group. 

The  offered  load  for  the  class-.s  traffic  for  node  pair  — >■  l.n2.x2)  is  A°(l.ni.:ri  —>■ 

l.n2.x2),  and  each  route  between  peer  groups  (second-layer  route)  gets  a  portion: 


Ajs(7>2(l.ni.yi  ->■  l.n2.yi))  =  a1\°s{l.ni.xi  ->■  l.n2.x2) 


7 


A°s(P2(l.ni.'i/2  -»■  l.n2.y2))  =  a2A2(l.ni.a:1  ->■  l.n2.a;2) 


->■  l-«2-y»»p))  =  anp\°s(l.n1.x1  ->■  l.n2.a:2) 

where  a*,?'  =  1,2,  ...,np  is  the  fraction  of  traffic  going  through  each  of  the  valid  route  set. 
1- 

For  simplicity  purposes,  denote  2 .y,-,  as  any  node  on  the  second  layer. 

So  the  aggregated  traffic  for  node  pairs  on  the  second  layer  is 

->  2.Vj)  =  Y.  ■4U’2). 

wiinw;) 

We  thus  have  all  the  input  traffic  load  for  the  second  layer  and  the  fixed  point  method  for 
a  fiat  network  with  dvnamic/state  dependent  routing  can  be  applied  [7].  This  results  in  the 
end-to-end  blocking  probability  Bs(2.y,i  — >  2.y?). 

As  discussed  earlier,  different  criteria^ delay,  blocking  probability,  implied  cost,  available 
bandwidth)  associated  with  the  second  segments  of  the  same  original  traffic  flow  should 
match  the  distribution  of  traffic  flow  onto  these  segments.  Ultimately  one  of  the  goals  for 
any  dynamic  routing  scheme  is  to  balance  traffic  load  on  different  alternative  routes,  and 
the  end  result  is  that  these  alternative  routes  will  have  equivalent  QoS  under  steady  state. 
For  example,  we  can  use  blocking  probability  as  a  criteria  to  adjust  the  traffic  distribution 
<iiy  i  =  1,2,...,  np.  Segments  with  a  blocking  probability  higher  than  median  gets  a  decreased 
portion,  and  segments  with  a  blocking  probability  lower  than  median  gets  an  increased 
portion: 

a i  :=  cii  +  S  if  Bsil.n-i_.yi  — >  l.n2.y,)  <  Bm ; 
a-i  :=  CH  -  5  if  Bs{l.ni.yi  — >  1- n2.y,)  >  Bm> 

where  S  is  a  small  incremental  value  and  Brn  is  the  median  blocking  probability  among 
all  routes.  Other  means  of  relating  traffic  distribution  to  route  QoS  can  also  be  specified. 
Another  round  of  iteration  is  then  started  using  these  new  distribution  values.  This  process 
continues  until  all  routes  have  similar  blocking  probabilities. 


4.3  Lower  Layer  Relaxation 

From  the  offered  load  distribution  calculated  from  the  higher  layer  relaxation,  we  now  have 
complete  traffic  information  for  each  peer  group,  including  the  traffic  when  the  group  is  a 
source  group,  a  destination  group  or  an  intermediate  group.  The  reduced  load,  which  is  the 
above  thinned  by  blocking  on  the  second  layer  and  the  remote  peer  group,  becomes  the  input 
offered  load  for  calculations  on  this  layer  in  the  same  way  to  that  in  the  fixed  routing  model. 

Iteration  is  done  in  a  similar  way  to  that  with  higher  layer.  This  will  result  in  a  new  set 
of  values  of  traffic  distribution,  which  is  then  used  by  the  next  iteration  on  the  second  layer. 
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5  Numerical  Results 


We  have  run  numerical  experiments  for  the  network  example  shown  in  1  using  fixed  hierar¬ 
chical  routing  scheme.  This  is  a  21-node,  30-link,  3-clusters,  2-layer  network  model.  We  used 
single  class  of  traffic  requiring  unit  bandwidth.  Link  capacity  varies  between  60,  80,  100  and 
120  units  of  bandwidth.  Due  to  space  limit,  detailed  offered  traffic  rates  and  link  capacities 
are  not  listed  here  but  can  be  found  in  [11],  Below  is  a  comparison  between  flat  fixed- 
point  approximation  and  hierarchical  fixed-point  approximation  on  individual  link  blocking 
probabilities  (end-to-end  blocking  probabilities  are  computed  directly  from  these  for  fixed 
routing).  We  observe  a  near  4-fold  improvement  in  computational  cost.  Experiments  and 
simulation  for  the  case  with  dynamic  hierarchical  routing  are  in  progress. 


link 

Hierarchical  FPA 

Flat  FPA 

(1.1. 7-1. 1.2) 

0.0000 

0.0000 

(1.1. 1-1. 2.1) 

0.4880 

0.4823 

(1.2. 2-1.3. 1) 

0.0514 

0.0515 

(1.3. 1-1. 3. 4) 

0.0391 

0.0394 

(1.3. 3-1. 3. 5) 

0.0007 

0.0007 

time  (sec) 

11.23 

40.88 

6  Conclusion 


1  In  this  paper  we  presented  a  hierarchical  reduced  load  approximation  scheme  for  networks 
with  either  fixed  hierarchical  routing  or  dynamic  hierarchical  routing  policies.  This  is  a  novel 
approximation  method  for  efficient  and  scalable  performance  analysis.  It  can  also  be  used 
in  cases  where  different  routing  schemes  are  used  in  different  regions  of  a  network.  Our 
preliminary  numerical  experiment  results  showed  significant  improvement  in  computational 
cost. 
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